Skip to content

feat(network): support --aux-address on network create#5041

Open
mayur-tolexo wants to merge 1 commit into
containerd:mainfrom
mayur-tolexo:feat/network-aux-address
Open

feat(network): support --aux-address on network create#5041
mayur-tolexo wants to merge 1 commit into
containerd:mainfrom
mayur-tolexo:feat/network-aux-address

Conversation

@mayur-tolexo

@mayur-tolexo mayur-tolexo commented Jul 4, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Part of #5012.
Docker's network create reserves auxiliary addresses so IPAM never hands them out to containers. nerdctl had no equivalent.

host-local has no exclude list, but it allocates across every range in a set, so each reserved IP is carved out by splitting the subnet range around it. The reserved name=IP pairs are matched to the subnet that contains them (so dual-stack picks the right family), rejected when they hit the network or gateway address or fall outside every subnet, and recorded so network inspect reports AuxiliaryAddresses the same way Docker does.

@mayur-tolexo mayur-tolexo marked this pull request as draft July 4, 2026 09:08
@mayur-tolexo mayur-tolexo force-pushed the feat/network-aux-address branch from 08a26ea to d6ab811 Compare July 13, 2026 06:32
@mayur-tolexo

mayur-tolexo commented Jul 13, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor Author

Checked --aux-address against Docker 29.4.0 to make sure the behaviour and output line up. Reserved addresses show up in network inspect the same way, an aux-address that matches no subnet or lands on the network/gateway address is rejected the same way, and dual-stack keeps each family's aux-address on the right subnet.

Docker:

$ docker network create --subnet 10.50.0.0/24 --aux-address host1=10.50.0.5 --aux-address host2=10.50.0.6 auxnet
$ docker network inspect auxnet --format '{{json .IPAM.Config}}'
[{"Subnet":"10.50.0.0/24","Gateway":"10.50.0.1","AuxiliaryAddresses":{"host1":"10.50.0.5","host2":"10.50.0.6"}}]

$ docker network create --subnet 10.51.0.0/24 --aux-address bad=192.168.9.9 badnet
no matching subnet for aux-address 192.168.9.9

$ docker network create --subnet 10.52.0.0/24 --gateway 10.52.0.1 --aux-address g=10.52.0.1 gwnet
Error response from daemon: failed to allocate secondary ip address (g:10.52.0.1): Address already in use

$ docker network create --ipv6 --subnet 10.55.0.0/24 --aux-address v4=10.55.0.9 --subnet fd00:55::/64 --aux-address v6=fd00:55::9 dualaux
$ docker network inspect dualaux --format '{{json .IPAM.Config}}'
[{"Subnet":"10.55.0.0/24","Gateway":"10.55.0.1","AuxiliaryAddresses":{"v4":"10.55.0.9"}},{"Subnet":"fd00:55::/64","Gateway":"fd00:55::1","AuxiliaryAddresses":{"v6":"fd00:55::9"}}]

nerdctl with this PR:

$ nerdctl network create --subnet 10.50.0.0/24 --aux-address host1=10.50.0.5 --aux-address host2=10.50.0.6 auxnet
$ nerdctl network inspect auxnet --format '{{json .IPAM.Config}}'
[{"Subnet":"10.50.0.0/24","Gateway":"10.50.0.1","AuxiliaryAddresses":{"host1":"10.50.0.5","host2":"10.50.0.6"}}]

$ nerdctl network create --subnet 10.51.0.0/24 --aux-address bad=192.168.9.9 badnet
level=fatal msg="no matching subnet for aux-address 192.168.9.9"

$ nerdctl network create --subnet 10.52.0.0/24 --gateway 10.52.0.1 --aux-address g=10.52.0.1 gwnet
level=fatal msg="failed to allocate secondary ip address (g:10.52.0.1): Address already in use"

$ nerdctl network create --ipv6 --subnet 10.55.0.0/24 --aux-address v4=10.55.0.9 --subnet fd00:55::/64 --aux-address v6=fd00:55::9 dualaux
$ nerdctl network inspect dualaux --format '{{json .IPAM.Config}}'
[{"Subnet":"10.55.0.0/24","Gateway":"10.55.0.1","AuxiliaryAddresses":{"v4":"10.55.0.9"}},{"Subnet":"fd00:55::/64","Gateway":"fd00:55::1","AuxiliaryAddresses":{"v6":"fd00:55::9"}}]

host-local has no exclude list, so the reservation is enforced by splitting the range around each reserved IP; those addresses are then never handed out. With --aux-address host1=10.50.0.5 --aux-address host2=10.50.0.9 the generated host-local ranges skip .5 and .9:

rangeStart 10.50.0.1   rangeEnd 10.50.0.4
rangeStart 10.50.0.6   rangeEnd 10.50.0.8
rangeStart 10.50.0.10  rangeEnd 10.50.0.254

@mayur-tolexo mayur-tolexo force-pushed the feat/network-aux-address branch 2 times, most recently from 4869592 to 96aef29 Compare July 13, 2026 07:01
@mayur-tolexo
feat(network): support --aux-address on network create
8400fe2 
Docker's network create reserves auxiliary addresses so IPAM never hands
them out to containers. nerdctl had no equivalent.

host-local has no exclude list, but it allocates across every range in a
set, so each reserved IP is carved out by splitting the subnet range around
it. The reserved name=IP pairs are matched to the subnet that contains them
(so dual-stack picks the right family), rejected when they hit the network or
gateway address or fall outside every subnet, and recorded so network inspect
reports AuxiliaryAddresses the same way Docker does.

Signed-off-by: Mayur Das <[email protected]>
@mayur-tolexo mayur-tolexo force-pushed the feat/network-aux-address branch from 96aef29 to 8400fe2 Compare July 13, 2026 07:03
@mayur-tolexo mayur-tolexo marked this pull request as ready for review July 13, 2026 07:10
@mayur-tolexo

mayur-tolexo commented Jul 13, 2026

Copy link
Copy Markdown
Contributor Author

cc @AkihiroSuda — rebased on latest main and green, could you take a look when you get a chance?

@AkihiroSuda AkihiroSuda added this to the v2.4.0 milestone Jul 13, 2026
AkihiroSuda
AkihiroSuda reviewed Jul 14, 2026
View reviewed changes
Comment thread pkg/netutil/cni_plugin.go
// host-local does not read it (reservation is done by splitting the range
// around each IP); it is nerdctl bookkeeping so `network inspect` can
// report AuxiliaryAddresses the way Docker does.
AuxiliaryAddresses map[string]string `json:"auxiliaryAddresses,omitempty"`

@AkihiroSuda AkihiroSuda Jul 14, 2026 •
edited
Loading

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doesn't seem implemented in CNI
https://github.com/containernetworking/plugins/blob/v1.9.1/plugins/ipam/host-local/backend/allocator/config.go#L68

nerdctl-specific information could be stored in NerdctlLabels
https://github.com/search?q=repo%3Acontainerd%2Fnerdctl+NerdctlLabels&type=code

@AkihiroSuda AkihiroSuda Jul 14, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Found a similar bug

AkihiroSuda
AkihiroSuda requested changes Jul 14, 2026
View reviewed changes

@AkihiroSuda AkihiroSuda left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#5041 (comment)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@AkihiroSuda AkihiroSuda AkihiroSuda requested changes

Requested changes must be addressed to merge this pull request.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

v2.4.0

Development

Successfully merging this pull request may close these issues.

2 participants

@mayur-tolexo @AkihiroSuda